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TWO DIFFERENT APPROACHES TO STOCHASTIC RECURSIVE 

OPTIMAL CONTROL PROBLEM WITH DELAY AND 

APPLICATIONS* 

JINGTAO SHit 

Abstract. This paper is concerned with one kind of stochastic recursive optimal control problem 
with time delay, where the controlled system is described by a stochastic differential delayed equation 
(SDDE). The cost functional is of the recursive utility' type, which is formulated as the solution 
to a backward stochastic differential delayed equation (BSDDE). Two approaches to study such 
vN , problem are Bellman's dynamic programming and Pontryagin's maximum principle. When only the 

5^ 1 pointwise and distributed time delays in the state variable, and only distributed time delay in the 

^ l' generator of BSDDE, a generalized Hamilton-Jacobi-Bellman equation in finite dimensional space is 

.^^ ' obtained applying the dynamic programming approach and a generalized fto's formula. Sufficient and 

^^ ' necessary maximum principles are both derived, where the adjoint equation is a forward-backward 

f-f^ , stochastic differential delayed equation (FBSDDE). Moreover, the relationship between these two 

^\J 1 approaches are investigated. Under some differentiability assumptions, connections among the value 

function, the adjoint processes and the generalized Hamiltonian function are investigated. 

A consumption and portfolio optimization problem with recursive utility and bounded memory 
rj ^ in the financial market, is discussed to show the applications of our result. Explicit solutions in a 

finite dimensional space derived by the two different approaches, coincide. 
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^ . 1. Introduction. The research of many natural and social phenomena shows 

Cn I that the future development of many processes depends not only on their present 

state but also essentially on their previous history. Such processes can be described 

vQ ' by stochastic differential delayed equations (SDDEs). Many examples can be found in 

Mohammed [20l [21] . Stochastic optimal control problems with time delay are those 
whose dynamics of states are described by SDDEs, and to find some optimal control 

^^ . to maximum/minimize the corresponding cost functionals or utility functions. In 

general, stochastic optimal control problems with time delay are practically intractable 
because of the complex infinite-dimensional state space framework. 

However, in certain cases, stochastic optimal control problem with time delay 

1^^ ■ can be reduced to a finite-dimensional one and solved explicitly. To the best of 

S I our knowledge, the first example of such a solvable problem is a linear delayed system 

" " " with a quadratic cost functional, given by Kolmanovskii and Maizenberg [16] (see also 

Kolmanovskii and Shaikhet [17.), where only the pointwise and distributed time delays 
are involved in the state variable (see (|1.2p in this section). Then Elsanosi et al. [TT] . 
Larssen and Risebro |19| extend Bellman's dynamic programming principle to time 
delayed case, where the initial problems are reduced to finite-dimensional Markovian 
systems and then obtain suitable versions of the Hamilton-Jacobi-Bellman equations 
(HJB equations). Then, their applications to optimal harvesting was considered in 
Elsanosi et al. 11 , to optimal consumption problems under partial observation was 
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2 STOCHASTIC RECURSIVE DELAYED OPTIMAL CONTROL 

studied in Elsanosi and Larssen ^10] . and to stochastic portfolio optimization model 
with bounded memory was studied in Chang et al. [J] recently. 

Along another line, 0ksendal and Sulem [2 2) apply Pontryagin's maximum princi- 
ple to discuss stochastic optimal control problems with time delay, where also only the 
pointwise and distributed time delays are involved in the state variables. Sufficient 
conditions for optimality are given by the Hamiltonian function and three adjoint 
equations, two of which are backward stochastic differential delayed equations (BSD- 
DEs) and the other one is a backward ordinary differential delayed equation (BODDE). 
Stochastic optimal control problems with time delay by the maximum principle ap- 
proach, can also be seen in Chen and Wu [5|, 0ksendal et al. [23], Agram et al. [1] 
and Yu [31], where anticipated or time- advanced BSDEs of Peng and Yang [5S]'s type 
are used (note that there are also time delays in the control variable in [S], [M])- 

Now, let us present the stochastic optimal control problem with time delay. Let 
{W{t),t > 0} be a one-dimensional Brownian motion on a complete probability space 
(ri, -F, P). We assume that the filtration !F^ = a{W{T); s < r < t} is augmented by all 
the P-nuU sets in iF. Let < T < oo be the fixed time duration and Q < 5 < oohe the 
constant time delay. Denote C([— 5, 0];R) the Banach space of continuous functions 
7 : [— (5, 0] -^- M with norm ||7||c :— sup |7(i)|. 

-i5<i<0 

For given initial time s G [Q.T), the dynamics of the state is described by the 
following SDDE 

dX'''^'"(t) = b{t, X'''^'"(t), Xi"''^'"(t), X2'''^'"(t), U(t))dt 

+ a(i, X^-^^"(i), Xi^'^^"(t), X2^^^^^(i), u{t))dW{t), t e is, T], (1.1) 
X^''^'"(t) = (^(f-s), te [s-6,s]. 

Here continuous function (p : [~S, 0] -^ M is the initial path of Ar*''^'"(-). Let U C M be 
a nonempty convex set. Control process u : fl x [0, T] — > U is an J^f -adapted process 
and 

.0 

^r'^'"W=/ e^^X''^'"(t + T)dT, X^''^'-''{t)^X'^'f'''{t~S), (1.2) 

represent given functionals of the path segment X^'"^'^ := {X''-'^'^{t + t);t E [—S, 0]} 
of X*'''^'"(-). A e R is the given averaging parameter, b : [0, T] x K"^ x U — > M and 
cr : [0, T] X M'^ X U ^- M are given continuous functions. 

For given (s,(p) S [0,r) xC([-(5, 0];R), we denote Z^[s,T] as the set of controls m(-) 
with the property that if u(-) eU[s,T] then (|l.ip admits a unique J^^^^-adapted solution 
A:''''^'"(-). We refer to such -u(-) G U[s, T] as an admissible control and (u(-), X"^'^'"(-)) 
as an admissible pair. 

Let / : [0, r] X M^ X U — > M and /i : M^ — > M be given continuous functions. For 
given (s, (p) £ [0, T) x C{[—S, 0]; R) and admissible control u{-) G U[s, T], consider the 
following cost functional 

J(s, <^; ui-)) - e| ^ fit, X^''^-^^{t),Xl^^--'-{t),u{t))dt + /i(X^-^^"(T), Xi^'^^"(T)) 

(1.3) 
Problem (SOCPD). The stochastic optimal control problem with time delay 
is to find u*{-) eU[s,T], such that 

V{s,^)^J{s,^;u*{-))= -mi J{s,p;u{-)), (1-4) 
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forall(s,(p) G [0,r)xC([-(5,0];R),withl/(T,(/j) ^ h{ip{0)),ip e C{[-S,0];R). Such a 
control process u*{-) is called an optimal coniro/ of Problem (SOCPD) (if it exists). 
The corresponding solution X^'''^''"{-) is called the optimal state and V{s,Lp) is called 
the value function. 

Problem (SOCPD) and its applications in economics and finance has been 
studied by many authors. Among others, Larssen [18^ and Larssen and Risebro [19] 
established the dynamic programming principle for this problem, and then obtained 
the corresponding HJB equation in a finite dimensional space. Under certain condi- 
tions, they prove that the value function is the unique viscosity solution to the HJB 
equation. 0ksendal and Sulem j22j apply the maximum principle approach to obtain 
some sufficient conditions for optimality, by the Hamiltonian function and three ad- 
joint equations, in which two of them are BSDDEs and the other one is a BODDE. 
The relationship between these two approaches is studied recently by Shi [28]. Un- 
der the assumption that the value function is enough smooth, connections among its 
derivatives, the adjoint processes and the generalized Hamiltonian function are given. 

In this paper, we study this kind of stochastic optimal control problem with time 
delay, in the case when the cost functional J{s, </?; u{-j) (|1.3|1 is replaced by a functional 
of recursive utility type. As found by Chen and Wu [6. , this kind of utility functional 
can be regarded as the solution to a BSDDE, which is a natural generalization of 
classical BSDE to time delayed case. For more information about stochastic optimal 
control problems with recursive utility functionals and their applications, please refer 
to Duffie and Epstein , El Karoui et al. [SI |9] , Schroder and Skiadas |26] , Wu and 
Yu [32], Wang and Wu [30], Shi and Wu [29], Wu [31] and the references therein. 

More precisely, we introduce the following controlled BSDDE coupled with the 
above SDDE (HT]): 

-dY'''p--"{t) = f{t, x*^'^'"(i), ^'''^'"(i), r*'^'"(i), ^"''^'"(t), u{t))dt 

- z*''^'^{t)dw{t), ie[s,r], (1.5) 

ys,v;«(2.) ^ ,/,(A:"^'^'"(r),Xi"''^'"(r)). 

Here / : [0, T] x M^ x r2 x U ^ R, : M" ^ M are given functions. 

For given {s,ip) G [0,r) x C([-J, 0];R) and admissible control w(-) G W[s,r], the 
recursive utility functional is defined as follows 

Ms,v;u{-)) = -Y-^^^-'-{t)l^^. (1.6) 

Here U[s,T] is defined as the set of controls u(-) such that if u(-) 6 U[s,T] then 
SDDE drU and BSDDE ([131) admit unique J^^'-adapted solutions X-^'f-'^i-) and 
(y^^¥';"(-),Z^'¥';"(.)), respectively, for all {s,ip) G [0,r) x C([-(5, 0];M). 

Problem (SROCPD). The stochastic recursive optimal control problem with 
time delay is to find u*{-) G U[s,T], such that 

Vris,ip) = Jr{s,(p;u*{-)) ^ essinf Jr{s,ip;u{-)), (1.7) 

u(-)e U[s,T] 

for ah {s,ip) G [0,T) x C([-(5, 0]; R), with Vr{T,ip) = -0((p(O)),^ G C([-J,0];R). 

This problem can be reformulated as follows. The state process (X''''^'"(-),y''^'"(-), 



(1.8) 
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Z*'''''"(-)) of our system is described by the following coupled FBSDDE 

' dX''''^'"(i) = b{t, X''''^'"(i), Xi"'^'"(i), X2'''^'"(i), U{t))dt 

+ a{t, X^^'^'"(t), Xi^''^'"(i), X2^''^'"(i), M(t))dVK(i), 

-dY'''^--"{t) = f{t, x*''^'"(i), x*''^'"(t), r"''^'"(i), ^"''^'"(i), uit))dt 
X^'^-"(i) = ^(t-.s), te[s~6,s], 

ys,v;«(2.) ^ ,/,(x"^'^'"(r), x;'''^'"(r)), 

and the cost functional is given of the form 

Here E*''^'"[-] denotes expectation with respect to the law of X*''^'"(-). 

Problem (FBSOCPD). The forward-backward stochastic optimal control prob- 
lem with time delay is to find u*{-) £ U[s,T], such that 

K(s,(/3) = Jr(s,V5;w*(-)) = essiiif Jr(s,V5;w(-)), (1-10) 

u(-)£ W|s,TJ 

for ah (s,(/3) G [0,T) x C([-^,0];R), with VriT,(p) = -</)((/3(0)), </? S C([-5,0];]R). 

To our best knowledge, there are only a few papers on forward-backward stochas- 
tic optimal control problems with time delay (see Fuhrman et al. [13], Huang et al. 
[H], Shi [27], Huang and Shi [lS], Chen and Wu [6^, Agram and 0ksendal [2])- In 
Fuhrman et al. |13j . under infinite dimensional space framework, for some special 
case of optimal control problem of FBSDDEs, the value function is proved to be a 
mild solution to the corresponding HJB equation and the existence of optimal con- 
trols in the weak sense is proved. In Huang et al. 1141 , explicit optimal controls 
are obtained for some kind of linear anticipated FBSDDE by Riccati equations. In 
Shi '275, maximum principles for optimal control of anticipated FBSDDE with time 
varying delay and Poisson jumps are proved. In Huang and Shi [15,, both necessary 
and sufficient maximum principles for optimal control problem for one kind of fully 
coupled anticipated FBSDDE are derived. In Chen and Wu [6], stochastic recursive 
optimal control problem of SDDE in a general form is considered and the dynamic 
programming principle is presented. However, the value function is proved to be the 
viscosity solution to the corresponding infinite dimensional HJB equation, and no 
examples with explicit solutions are presented. 

Recently, the optimal control problem of an infinite horizon system governed by 
a FBSDDE with pointwise and distributed time delays was studied, by Agram and 
0ksendal ^. Sufficient and necessary maximum principles for optimal control under 
partial information in infinite horizon are obtained. An optimal consumption problem 
with respect to recursive utility from a cash flow with delay was discussed as their 
applications. However, since in their paper the adjoint backward equation is described 
as an anticipated BSDE of Peng and Yang [3S]'s type, no explicit solution is given 
(note that a solvable special case is given only when there is no time delay in [5^). 
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True, the solution to a stochastic (recursive) optimal control problem with time 
delay will depend on the initial condition (/?, which is in an infinite dimensional space 
C([— 5, 0];M). However, if the system of SDDE (|l.ll) depends only on the time delay 
through processes Xi{-) and X2{-) defined by (II. 2p (that is, the system involve only 
the pointwise and distributed time delays in the state variable), and the generator / of 
BSDDE (jl.Sp depends only on {X{-),Xi{-)) not X2(-), the problem can be reduced to 
a finite dimensional one and it is possible to obtain the solution in a finite dimensional 
space. This is the first main contribution of this paper, and we will make this point 
clear in Section 2. 

The other main contribution in this paper is that we initiate to study the relation- 
ship between Bellman's dynamic programming and Pontryagin's maximum principle 
approaches, for stochastic recursive optimal control problems with time delay. Such a 
topic is of great importance in stochastic control theory, see the systematic monograph 
by Yong and Zhou 33, and the references therein. Since the relationship between these 
two approaches is the one between the derivatives of the value function and the ad- 
joint processes along the optimal state, or actually the one between HJB equations 
and stochastic Hamiltonian systems, and more generally, the one between partial dif- 
ferential equations (PDEs) and stochastic differential equations (SDEs). Thereby, is 
is natural to ask the question: Are there any relations between these two extensively 
used and important approaches, for stochastic optimal control problems with time de- 
lay? The answer should be yes. However, to the best of our knowledge, results on this 
topic are quite lacking in literature. One main difficulty and obstacle is that the solu- 
tion to a stochastic optimal control problem with time delay, or controlled system with 
SDDE, will be in an infinite dimensional space framework. For generalization of Bell- 
man's dynamic programming principle to time delayed case, see Larssen |18j . Chang 
et al. [3] , Federico [T^] , where several versions of infinite dimensional HJB equations 
have been used. Moreover, their solvability in the infinite dimensional spaces is com- 
plicated and consequently their real applications are largely limited. Along another 
line, with the establishment of the theory of anticipated BSDEs by Peng and Yang 
[25j , Chen and Wu [5] developed necessary and maximum principles for the controlled 
system involving time delays in both the state variable and the control variable. And 
their results are applied to a production and consumption choice problem and explicit 
optimal consumption rate is obtained for some special case. 

In this paper, we consider the stochastic recursive optimal control problem with 
time delay (|1.8p ~ (|1.10p . For this problem, in Section 2, under some suitable as- 
sumptions, we first establish the dynamic programming principle for this problem 
and obtain the corresponding generalized HJB equation in a finite dimensional space. 
These results are natural corollaries of those in Chen and Wu [6j . Then we derive the 
sufficient and necessary maximum principles for the optimal controls. The adjoint 
equation that we introduce in this paper consists of one SDDE, two BSDDEs and 
one BODE, which is different from the one in Agram and Oksendal |2j. In Section 3, 
we investigate the relationship between the above two approaches: dynamic program- 
ming principle and maximum principle. Under the assumption that the value function 
is enough smooth, we obtain the relations among its derivatives, the adjoint processes 
and the generalized Hamiltonian function. A consumption and portfolio optimiza- 
tion problem with recursive utility and bounded memory in the financial market is 
discussed in Section 4, to show the applications of our result. Explicit solutions in a fi- 
nite dimensional space derived by the maximum principle and dynamic programming 
approaches, coincide. Finally, Section 5 gives some concluding remarks. 
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2. Preliminaries, MEDcimum Principle and Dynamic Programming. In 

this section, under some suitable assumptions, we first establish the dynamic pro- 
gramming principle and present the corresponding HJB equation in finite dimensional 
space. Then we derive sufficient and necessary maximum principles for the optimal 
controls. Our results gain enlightenment from the work by Chen and Wu [6 , Agram 
and 0ksendal i2j. 

For any s G [0,r), the following notations are used in this paper. 

L'^{fl,J-^;R) :— < M-valucd 7^|n-measurable random variables ^; E|^p < cx) >, 

53r([s,T];R) := I M-valued J'l-adapted processes V'(^); E sup \^p{t)\ <ooi, 
'- '-s<t<T -I J ' 

L3r([s,T];K) := JR-valued J'l-adapted processes ip{t); E / |-0(i)|^di < ooj, 
l?{n;C{[-S,0];M.);T^) := jv- : ^2 ^ C([-5,0];M) is J"^measurable; the norm 

I</'IIl2(O;C([-5,0];R)) —E ||¥'(w)||c([_5^o];R) <Oo}- 

First, in order to ensure the existence of the unique solution to the SDDE (|l.ip . 
we introduce the following assumptions. 

(HI) The functions h{t,x,xi,X2,u) and (T(t, a;, a;i,a;2, u) are joint continuous and 
globally Lipschitz in (a;, Xi, 2:2)1 that is, there exists a constant C > such that 

\b{t,x,xi,X2,u) - b{t,x,xi,X2,u)\ + \a{t,x,Xi,X2,u) - a{t,xi,y2,z,u)\ 

< C{\x - x\ + \xi - Xi\ + \X2 - X2\), 

for all t e [0,T],x,x £R,xi,xi G R, 2:2 , i;2 eR and u e U. 
(H2) There exists a constant C > such that 

\b{t,x,xi,X2,u)\ + \a{t,x,xi,X2,u)\ < C(l + |a;| + |a;i| + \x2\), 

for aU t e [0, T], a; e M, xi G K, 2:2 e M and ueV. 

(H3) The initial path ip belongs to the space L^(r2; C{[—5, 0]; M); J^|) of J^|-measurable 
elements in L2(J7; C([-(S, 0];E)), that is, (^ : fi ^ C([-(5, 0];]R) is J"|-measurable and 

\\v\\h{n:,c{l-s.,o]-M)) ■=^ ll'/'(w)||c([-<5,o];R) < oo- 

The following classical result can be seen in Mohammed [201 El] ■ 

Lemma 2.1. Under assumptions (H1)^(H3), for any {s,ip) € [0,r)xL2(J7; C([-(5, 0];M)) 
and u{-) G U[s,T], the SDDE lil.l\) admits a unique adapted strong solution 

X^'^^«(.)eL2(r!;C([-5,0];M);J-,^). 

Next, in order to ensure the existence of the unique solution to the BSDDE (|1.5p . 
we introduce the following assumptions. 

(H4) /(■, X, xi, y, z, u) is J^/-measurable, for all a; G R, xi e R and m G U. 

(H5) The functions f{t,x,xi,y,z,u) and 0(a;,a;i) are joint continuous and globally 
Lipschitz in (x,xi,y, z), that is, there exists a constant C > such that 

\f{t,x,xi,y,z,u) - f{t,x,xi,y,z,u)\ + |(/)(x,xi) - (f>{x,xi)\ 
< C{\x - x\ + \xi - xi\ + \y - y\ + \z - z\), 
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for all t e [0,T],x,x e R, xi,2:i e R, X2,X2 ER,y,y &R,z,z eR and u e U. 
(H6) There exists a constant C > such that 

|/(t,a;,a;i,0,0,u)| + \<l){x,xi)\ < C(l + |a;| + |a:i|), 

for all t e [0, T], a: e M, a;i e R and u e U. 

The following result can be obtained from the classical BSDE theory, by Lemma 
2.1. See also Chen and Wu 0. 

Lemma 2.2. Under assumptions (H1)^(H6), for any (s, ip) G [0, T) xl?{n; C{[-5, 0]; M)) 
d u{-) e lA[s,T], the BSDDE hl.5\) admits a unique adapted solution 



an 



(y'''^'"(-),^"''^'"(-)) e 5'^([s,T];R) X L^([s,r];R). 



Now, we turn to consider Problem (SROCPD) or the equivalent Problem 
(FBSOCPD). The first approach is the Belmann's dynamic programming principle. 
In general, the value function Vr{s, (p) defined in (|1.10p may depend on the initial path 
(p G C([— ^, 0];R) in a complicated way. From Theorem 3.7 of Chen and Wu 6 , we 
know that the value function satisfies the following generalized dynamic programming 
principle: 



ueu[s,T] Ljs 

+ T4(j,Xf^^^")]' Vse[s,T]. 

(2.1) 
Here Xp'^''' is the map Xf^"^'" : [-S, 0] ^ R defined by Xf''^'"(T) := X''^'^'"(s + r). If 
we define the operator A^ on Borel measurable function Vr : [0, T] x C{[—S, 0]; R) -^ R 

by 

A^Vris, if) := lim ^ ^ ' \ '^ li^V^^ (2.2) 

sis S -- S 

then by (|2.ip the value function Vr{s, ip) solves the following 

sup \ A^Vr{s, (p) + /(s, 1^, -Vr{s, ^) , -VoV;.(s, ^)cr{s, ^, u) , u)\ =0, 



V(s,^)e[0,T)xC([-,5,0]; 
Vr{T,ip) = -cj){ip), V^eC(h5,0];R), 



(2.3) 



where VoK-(s, (p) = 'V^Vr{s, (p){{0}) and VyT/r(s, ^) is the gradient of K- with respect 
to if at point (s, 1^9). This is a backward stochastic evolution equation (BSEE) in infinite 
dimension space (see Theorem 4.9 of Chen and Wu [6| for the derivation of BSEE 
(j2.3p ). However, looking at the recursive cost functional ()1.9p and ()1.10p . one might 
expect that the value function Vr{s, ip) depends on ip only through the two functionals 



X = x{ip) := ip{0), x\ = x\(ip) := e '^ip{T)dT. 





At, 



-5 



In fact if this is the case, the operator A^ defined by (|2.2p is a differential operator 
and the equation (|2.3p is a second order HJB partial differential equation (PDE) . This 
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is essentially due to the fact that there is an Ito's formula in this text. 

Lemma 2.3. (The Ito's formula) Letg £ C1'2'1([0,T]xR2), processes {X{-),Xi{-), 
X2{-)) are defined by lil.l\ ) and il.S^) . then for given u G U, we have 

dgit,X{t),Xi{t)) 

^ {^{t,X{t),X,{t)) + b{t,X{t),X,{t),X,{t),u)^{t,X{t),X,{t)) 

1 02 

+ -a^it,X{t),Xi{t),X2it),u)-^{t,X{t),X,it)) (2.4) 

+ [X{t)-XX,{t)-e-^'X2{t)]-^^{t,X{t),X,{t))\dt 

+ ait, X{t), Xi{t),X2it),u)^{t, X{t),Xi{t))dWit). 

ox 

The proof of this Ito's formula can be seen in Elsanosi et al. [11] or Elsanosi and 
Larssen [lO] . 

Theorem 2.4. (The Generalized HJB Equation) If the value function Vr for 
Problem (FBSOCPD) depends on (s,x,xi) only and Vr(s,x,xi) e Ci'2'i([0,r] x 
R^), then Vr{s,x,xi) solves the following generalized HJB equation 

( 9Vr, , ^/ ^,, , dVr, 

TT-(S,X,Xi) + SUpG[S,X,Xi,X2,U,-Vr{s,X,Xi), ;— (s, X, Xi ) , 

OS „eu ^ OX 

d'Vr,_^, dVr,_^,\_^ ^^ ^^ (2.5) 



OX'' axi 

, K(T,x,Xi) = -(/)(x,a;i), 



{s,x,xi),--—^{s,x,xi)\ =0, Va;: 



2 eiK, 



where the generalized Hamiltonian function G : [0,r] x M'^ x U x M'* — >■ M «s defined 
as 

G{s,x,xi,X2,u,r,p,R,q) :— b{s,x,xi,X2,u)p 

+ -a'^{s,x,Xi,X2,u)R+ (x - Xxi - e^^^X2)q (2.6) 

+ /(s, X, xi , r, a{s, x, Xi , X2,u)p, u). 

Proof Fix (s,x,xi) G [0,r] x R^ and u G U. Let ^"■^■=^i;"(-) be given by SDDE 
(|LT|) with u{t) = u, and fix ^ G C([-^,0];R) such that x = x{ip) = X"'^'^i'"(s) and 
xi = xi{ip) = ^"■'^'''^'"(s). And let (y^.^>^i;"(.)^ z^.^>^i;«(.)) be the corresponding 
solution to BSDDE ([T3)) . Then (PH implies that 

Vr{s,X,Xi) 

(2.7) 



- inf E"'^^^!'" 



/^■-■-i'"(t)dt + V;(s,Xf'"'"^'") , VsG[s,T], 



where for notations simplicity, we use (and will use in this proof) 

j:s,x,c,,-u^^^ = /(i, ^"■^■=^i'"(t), Xi'^'''''l'"(i), y'''="'^i'"(t), Z'*'^'="i'"(i), M(i)) , 
^s,x,x^■u^^^ ^ ^(^^ X^^^'^i'"(i), Xi"'="'^^'"(i), X|'="'^^'"(t), M(i)). 
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Comparing (I2.7P with (|2.1I) . we have replaced the essential infimum "essinf by the 
infimum "inf. Since by Proposition 3.2 of Chen and Wu [B], we know that the value 
function Vr{s,x,xi) is now a deterministic function. 
First, (pJj) implies that 



Vr{s,x,xi) <_E«'"'"i'"[ / /^■"■=^i'"(i)dt + K(s,Xf'"^"i^")], VsG [s,T]. 
Applying the Ito's formula (Lemma 2.3) and with s ]. s, we obtain that 



lim^^ E 

Sis S — S 






BV 



di 



— — (S, X, Xi) + &(S, X, Xi , X2, u) — — (s, X, Xi) 

OS ox 

+ -CT (s,x,xi,a;2,M)-^-2-(s,a;,a;i) + [x- Axi -e X2J ^ — [s,x,xi) 

^.^ E[Vr{s,Xl-''-'^)]-K{s,^) 
sis s — s 



> lim ■ 



1 



-E" 



s,x,Xi;u I rs,x.Xi;u 



Sis S — S 

Then by ^^ and ^^, we have 



/'''^'^i'"(t)di. 






Z'*'^'^i:"(s) 



9x 



(s,x,xi)cr(s, x, a;i,X2,u), Va;2 G ^ 



(2.8) 



Thus, 



— — (S, X,Xi)+ b[S, X, Xi,X2, u)——{s, X, Xi) 

OS ox 

1 2/ ^(^^Vr , X r A -A^ l^Vr, , 

+ -a [s,x,xi,X2,u)—-^(s,x,xi) + [x ~ Xxi - e X2\^ — {s,x,xi) 
— /(s,x,xi, — y,.(s,x,xi), — ^(s,a;,a;i)cr(s,x,Xi,X2,u),-u) > 0, Vx2 G M. 
This holds for any u G U, and by the definition of G (|2.6p . we arrive at 



dVr 
ds 



{s,X,Xi) + sup G(s,X,Xi,X2,U, -Vr{s,X,Xi), —-—^{s,X,Xi), 

"^" , ^" (2.9) 

- -^-^(s,x,Xi),-^-^(s, a;, a;i)J < 0, Va;2 e M. 
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Conversely, for any e > 0,0 < s < s < T with s — s small enough, we can find a 

u{-) = u^j{-) Eh([s,T] such that 

Vr{s, X, Xi) + e{s — s) 



or equivalently, 
-e< 



E[Vr{s,Xl'^---)]-Vr{s,v) , 1 



s — s 



s — s 



Again, it follows from the Ito's formula (Lemma 2.3) that 



-e < 



-E 



dVr 



(^^^s,.,.i;n(^)^^s,x,.i;«^^^^ 



OX 

+ f{t, X^^"^"i'"(i), Xi"'"'"^'"(t), -Vr{t, x^'^'^'-'^'it), Xi"'"^"i'"(t)), 



8V 



dt. 



Again, as s J, s, by the definition of G (I2.6p . we see that 

-e < — -— (s,x,xi) + G s,a;,a;i,X2,-u, -VV(s,x,xi), — -— (s,x,xi), 
OS V c*a; 



aK- 



(s,a;, xi),-— ^(s,x,xi)j 



< -- 



(s, x,xi) + sup G(s,x,Xi, a;2,u, — K(s, x,X\), — — -^(s,x,xi), 
«eu ^ OX 

— ^(s,a;, zi), -— ^(s,x,xi)j, Va;2 £ K. 



(2.10) 



Combining the inequalities (12. 9p with (|2.10l) gives (|2.5p . The terminal condition fol- 
lows immediately from the definition of the value function. The proof is complete. 
D 

The second approach to solve Problem (FBSOCPD) is the Pontryagin's maxi- 
mum principle. In this part, we let the initial time s — and write X = X" — X^''^''^, 
etc. Moreover, we need the following additional assumptions. 

(H7) The functions b{t,x,xi,X2,u),a{t,x,xi,X2,u) are continuously diiferentiable 
in {x,xi,X2,u), such that 



E 



db 



da 



/ OO ■^ 0(7 ^\ 

(— (i,X(t),Xi(i),X2(t),u(i)) + —{t,X{t),X,{t),X2{t),u{t)) jdiUoo, 
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for X — x,Xi,X2,u. 

(H8) The function f{t,x^xi,y,z,u) is continuously difFerentiable in {x,xi,y, z,u) 
and (j){x,xi) is continuously differentiable in {x,xi), such that 



E 



5/ 
dX 



it,X{t),Xi{t),Y{t),Z{t),u{t)) 



dt 



dx 



{X{T),X,{T)) 



dcj) 
dxi 



{X{T),X,{T)) 



< oo, 



for X = x,xi,y, z, u. 



We introduce the Hamiltonian function H : [0, T] x R^ x U x M'^ x M x M? 

H{t, X, xi,X2,y, z, u,p, q, k) := pih{t, x, xi,X2,u) 

+ P2{x — Xxi — e^^^X2) + kia{t,x,xi,X2,u) — qf(t,x,xi,y,z,u), 



as 



(2.11) 



where p= {pi,p2,P3y,k = (/ci,fc2)^. 

For each u*{-) e U[0,T] and the corresponding solutions {X* {■),¥*{■), Z*{-j) to 
(jl.l[) and (|1.5|) . we introduce the following adjoint equation 



-dpi{t) 

-dp2{t) 

-dpzit) 
dq{t) 

Pi{T) 
P2{T) 



dH* 

dx 
dH* 

dxi 
dH* 

8X2 

-{t)dt - —-{t)dw{t), t e [o,r], 



{t)dt-ki{t)dW{t), 
{t)dt-k2{t)dW{t), 
{t)dt, 



(2.12) 



dy 

84) 



dz 



dx 



iX*iT),XliT))q{T), 
{X*{T),X*,{T))q{T), 



dxi 

{ P3{T)^0, q{0)^l, 

where for notation simplicity, we use (and will use in the remaining part of this section) 

H*it)=H{t,X*it),X*,{t),X;it),Y*it),Z*{t),u*{t),p{t),q{t),k{t)), 

-^{t) = —{t,X*it),X:{t),X*2it),Y*it),Z*{t),u*it),m,q{t),k{t)), 

for p = x,xi,X2,y, z,u. Note that the above adjoint equation (|2.12p consists of two 
BSDDEs, one BODDE and one SDDE. From the definition H in (P?TT|) . under assump- 
tions (H7)'^(H8), there exists unique J^°-adapted solution {p{-) — {pi{-),p2{-),P3{-))'^ , 
(7(-),fc(-) = (fci(-),fc2(-))^)tothem. 

In recent work of Agram and 0ksendal [2], sufficient and necessary maximum 
principles for infinite horizon FBSDDE with Poisson jumps are obtained. In their 
paper, the backward part of the adjoint equation is described as a time advanced 
BSDE of 0ksendal et al. [25 's type, which is different from the adjoint backward 
part in (J2.12I) where three equations with respect to Pi{-) , P2{') and P3{-) are involved. 
The introduction of this type of adjoint backward equation comes from 0ksendal and 



12 STOCHASTIC RECURSIVE DELAYED OPTIMAL CONTROL 

Sulem ^2 . For practicable importance, our adjoint backward equations in (I2.12[) are 
more easier to be solved explicitly than the time advanced ones. 

Theorem 2.5. (Sufficient Maximum Principle) Let u*{-) G Uad, {X*{-),Y*{-), 
Z*{-)) be the corresponding solutions to il.l]} and U.5\} . andp{-) = {pi{-),P2{')tP?,{'))^ , 
q{-),k{-) — (fci(-)i fc2('))^ ^s ^^s corresponding solutions to \2.1Sfj) . Suppose that 



(x, xi,X2,y, z, u) -^ H{t, X, xi, X2, y, z, u,p{t),q{t), k{t)) is convex, 

for allte [0,T], 



(2.13) 



(x,xi) = Afx + iVxi, forM,N(^m., (2.14) 

P3(i)=0, /or o/ae[0,T], (2.15) 



— — (t)(u*(i)-u) <0, Vu e U, a.s., /or aZ/ie [0,r]. (2.16) 

Then u*{-) is an optimal control for Prohlem (FBSOCPD). 

Proof. For any u(-) G U[0,T], with the corresponding solutions {X {■),¥{■), Z{-)) 
to (|l.ip and (|1.5p . applying the Ito's formula to 

9(t)(r*(t)-r(i))+pi(t)(x*(t)-x(t)) 

.0 

+ P2{t) / _ e^"(X*(i + r) - X{t + T)dT+p3{t){X*{t -6)- X{t - 6)), 
noting that 

/O (.0 

e^^X{t + T)dT= X{t)-e-^^X{t-S)-X e^^X{t + T)dT dt, (2.17) 
-(5 L J-5 J 



we have 



E[q{T){Y*{T) - Y{T))] - [Y*{0) - Y{0)] 
-E[Mq{T){X*{T) - X{T))] -]K[Nq{T){Xl{T) - Xi{T))] 
= - [Y*{0) - Y{0)] = Jr{0, ip; u*i-)) - J,(0, ^; u{-)) 

= E^ |pi(t)(^^(t) _ ^{t,X{t),X,{t),X2{t),u{t)) 
+ h{t)(^^{t) - ^(t,X{t),X,(t),X2{t),u{t)) 

-q{t)(^^{t)~^{t,X{t),X,{t),Y{t),Z{t),u{t)) 

8h* r)rr* 

-pi{t)^{t){X*{t)-X{t))-k,{t) — {t){X*{t)-X{t)) 

+ q[t)-l-{t){X*{t) - X{t)) -p,{t) — {t){xm - X,{t)) 
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Bh* Bit* 

- Pi{t)g^^{t){x;{t) x,{t)) - k,{t)—{t){x;{t) x^{t)) 

+ q{t)^(t){Y*{t)-Y{t))+q{t)^{t){Z*{t)-Z{t))yt 
^eJ iH*{t) - H{t, X{t), X,{t),X2{t), Y{t),Z{t),u{t),p{t),q{t), k{t)) 

~ -^(t){X*^(t) - X^{t)) -^{t){Y*(t) - Yit)) 

-^-(t)[Z*{t)-Z{t))Yt- 
In the above, we have used 

^(i) = |^(i,x*(i),xr(i),xi(i),u*(i)), 

for p = x, xi, a;2, 2/, -z, «• By (|2.13p . that is i?(i, a;, xi, a;2, y, 2, u,p{t), q{t), k{t)) is con- 
vex with respect to (x, xi, 0:2, y, z, u), for all i G [0, T], we then have 

Jr{0,ip;u*{-)) - Jr{0,ip;u{-)) <E ——{t){u*{t) ~u{t))dt < 0. 

Jo c/u 

Thus u*{-) is really the optimal control for Problem (FBSOCPD). The proof is 
complete. D 

A drawback of the previous sufficient maximum principle in Theorem 2.5 is that 
the concavity conditions are not always satisfied in applications. In view of this, it 
is of interest to obtain conditions for an optimal control where the concavity is not 
needed. The following is a necessary maximum principle. For this target, we assume 
the following. 

(H9) For aU u e W[0, T] and aU bounded /3 £ U[Q, T], there exists e > such that 
u + e/3 gZ^[0,T], for aUe e (-e,e). 

(mo) For all to G [0, T] , /i such that to + h ^ [tf),T], and aU bounded 7^j° -measurable 
random variable a, the control process /3(i) defined by 

/3(t)=aX[*o.to+h](t), ie[0,T] (2.18) 

belongs to W[0,T], where X[to,to+h]{'t) is the indicator function on the set [toiio + h]. 
(Hll) For all bounded /? G Z//[0,T], the derivative processes 

m ■■= ^^"+^^WIs=o, (2.19) 
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C(i):-^Z«+-^(t)|e=o 



exist such that 

'■^ ''db 
dx 







{t,x{t),x^(t),x2{t),u{m{t) 



+ --{t, X{t), Xiit),X2it),uit)) / e^^^it + T)di 

db 

+ —{t,X{t),X^{t),X2{t),u{m{t - 5) 

db 

+ —{t,X{t),X^{t),X2{t),u{t))m dt 



—{t,x{t),x^{t),x2{t),u{t))m 

+ ^^{t,X{t),Xi{t),X2{t),u{t))l e^-at + r)d7 
+ ^^{t,X{t),Xi{t),X2{t),u{tmt - S) 



+ ^{t,x{t),x^{t),X2{t)Mt))m 

ou 



dt 



T V 



df 

Mt, X{t), X^{t),Y{t),Z{t),u{t))m 
ox 

+ ^{t,X{t),Xi{t),Y{t),Z{t),u{t)) f e^-i{t + T)dT 

OXi J_s 

df 
+ -^(t, X{t), X^{t),Y{t),Z{t),u{t))f^{t) 

df 

+ ^(i, X{t), X^{t),Y{t),Z{t),u{t))at) 



df 



{t,X{t),X,{t),Y{t),Z{t),u{t))m 



dt 



+ e| ^(x(r), Xi(r))e(T) + ^(x(r), Xi(T)) J°^ e^^^{T 



(2.20) 
(2.21) 



+ T)dT > < 00 



Note that 



|xr'»«)i„o = |(/;e-x-»«+.).. 



e=0 



° d 

e^-±lX"+"^(t + r) 
-5 de 
t 



dr 



e=0 



A(t-T) ^ yM+e/3 



(2.22) 



t-S 
t 



t-S 



de 



X"+^'^(t) 



dr 



e=0 



.Ht-r) 



ar)dT 



e^^^it + T)dT 
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and 



de 



X^+'^it) = -^X"+'f^{t-6) =^{t-S). (2.23) 



r=o de 



6 = 



It then follows from (ll.ip that the following variational forward equation 

dm = \-^{t,x{t),xm,x2{t),u{t))m 

+ — (i, X{t), Xi{t),X2{t), u{t)) I e^^^it + T)dT 

db 

+ — (i, X{t), Xi(i), X2(t), u{t))i{t - 5) 

+ ^{t,x{t),xm,X2{t)Mt))m\dt 

^{t,X{t),X,{t),Mt),n{tmt) ^^-^^^ 

ox 

^"^ {t,X{t),Xm,X2{t),u{t)) f e^^i{t + T)dT 



dxi 



6 



+ ^(i, X{t), Xi{t),X2{t), u{tmt - S) 

+ -£{t,Xit),Xiit),X2it),uit))/3it)\dWit), te[0,T], 

admits a unique J-"°-adapted solution ^(•), and the following variational backward 
equation 

~dv{t) = l^{t,X{t),Xm,Y{t),Z{t),u{tmt) 

+ ^{t,X{t),X,{t),yit),Z{t),u{t)) f e^^^{t + T)dT 
df 

+ ^(i, x(i), Xi(t), r(t), z{t), u{t))m 

+ g(t, x{t),x,{t), Y{t), z{t), u{t))at) ^^-^^^ 

+ ^(t, X{t),X,{t), Y{t), Z{t), u{t))P{t)\dt 
on J 

-(:{t)dW{t), ie[0,T], 
r,{T) ^ ^{X{T),X,{TmT) + ^^{X{T),X^{T)) J e^^^{T + T)dT, 

admits a unique J^°-adapted solution {ri{-), ({•))■ 

Theorem 2.6. (Necessary Maximum Principle) Let u*{-) £ U[0,T], and {X*{-), 
Y*(-), Z*{-)) be the corresponding solutions to il.l\) and il.5\) . adjoint processes p{-) = 
{Pi{')jP2{-),P3{-))^ ,q{t),k(-) = (fci(-), fc2(-))^ be the corresponding solutions to (KJM)- 
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The corresponding derivative processes (C*(')j ^*(')i C*(')) '^^'^ given by the variational 
forward and backward equations {2.24-^ and V2.25]) . Assume that 



E^ ^P?(t) 






Oct* , 2 



e''''C{t + T)dT 



-5 



Fin* 2 r)rr* 2 



+.-wif^w 



dt < (X). 



T/ien the following assertions are equivalent: 

(i) ttMu* +£l3) =0 for all bounded (3 e Z^fO, Tl; 



(2.26) 



(it) -^{t,x*it),x*{t),x;{t),Y*it),z*{t),u,m,q{t),k{t)) 



0, 



u=w{t) ' (2.27) 
a.s., /or aH i e [0, T]. 



Proof. Suppose that (i) holds. Then 

d 

= --Jr(0,x,xi;-u* + efH) 
oe 

Applying the Ito's formula to q{t)r]{t), we have 



6 = 



-r?(0). 



(2.28) 



q{T)[^{X*{T),Xl{T))C{T) 



+ §^^iX*(T), Xl{T)) J^^ e^^CiT + T)dT^ I - 77(0) 
= -E / lq{t)(^{t)C{t) + ?^{t) f e^^C{t + r)dT + ^{t)m 



/o I \ dx dxi 

Substituting (f2:29| into ^2B, we get 



du 



dt. 
(2.29) 



0= -E{q{T)[^{X*{T),X*(T))C{T) 

+ ^^iX*{T),X*iT))J e^^C{T + T)dT 



^ C {'^'^ (^^^^^*^^^ + 1£^*^ I] ''^^*^^ + ^^^" + ?f W/3W 



9u 



Applying again Ito's formula to 



dt. 
(2.30) 



Pi(t)r(i)+P2(t) / e^^C{t + r)dT+p,m*{t-S), 

1-5 
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and noting that 



e^^C {t + T)dT^ C it) - e-'T it-5)-X e^^r (t + T)dT 



dt. 



we have 



qiT)[^iX*{T),XUT))CiT) 



+ ^{X*{T),Xl{T)) I e^^C{T + T)dT\\ 



(2.31) 



Combining (I2.30p with (|2.3ip , noting that the definition of H in (|2.1ip , we get 

= E^ (^p,{t)^it,x*it),xm,x;{t),u*{t)) 

+ h{t)^{t,x*{t),xut),x;{t\u*it)) 

-q{t)^{t,X*{t),Xl{t),Y*{t),Z*{t),u*{t))^mdt 

= E ^{t)p{t)dt. 
Jo ou 

By (HIO), if we apply ([^3^ to 

where to G [0,r],/i such that t^ + h e [ioj^], s > to, a{uj) is bounded and J"(°- 
measurable, we get 



(2.32) 



E 



s+h 



dH* 

du 



{t)dt a 



0. 



Differentiating with respect to s, we obtain 

'dH 



E 



du 



(.) a 



0. 



Since this holds for all s > to and all J-"° -measurable bounded a, we conclude that 

'dH* 



E 



di 



-{to] 



-F" 



0, a.s. 



This shows that (i)=>(ii) by the adaptability. 

Conversely, since every bounded /3 G U[0,T] can be approximated by linear com- 
binations of controls (3 of the form (|2.18p . we can prove (ii)=4>(i) by reversing the above 
argument. The proof is complete. D 
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3. Relationship between Dynamic Programming Principle and Maxi- 
mum Principle. In this section, we investigate the relationship between dynamic 
programming principle (the generalized HJB equation) and maximum principle. The 
main result is this section is that we find the connection between the value function 
V, adjoint processes p, gand the generalized Hamiltonian function G, which is stated 
in the following theorem. 

Theorem 3.1. Let (Hl)^(Hll) hold and {s,x,xi) e [0,T)xR2 he fixed. Suppose 
thatu*{-) IS an optimal contro/ /or Problem (FBSOCPD) and {X* {■) ,Y * {■) , Z* {■)) 
is the corresponding optimal state which is the solutions to il-l]) and Hl.Sfl . Let 
P{') — {Pi{')iP2{'),P3(:))^ ,q{t),k{-) = (fci(-), fc2(-))^ be the solution to adjoint equa- 
tion 112. 12\) . Assum,e that the value function Vr depends on (s, x, xi) only andVr{s, x, xi) 



eCi^2,i([o,2^] 



then 



BV 

-^{t,X*{t),Xl{t)) 



dVr 



^G{t,X*{t),X*{t),X;it),u*{t),-Vr{t,X*{t),Xl{t)),—^{t,X*{t),X*,{t)), 



8^V dV 

-^{t,x*{t),x:{t)),~^^{t,x*{t),x*{t))) 

mi,^G{t,X*it),X*{t),X;it),u,~Vrit,X*it),X*{t)), 



dVr 

dx 



{t,x*{t),xm), 



rl'^V BV 

-^{t,X*{t),X*{t)),-^^{t,X*{t),X*{t))), a.e.te[s,T],a.s. 

(3.1) 

Further, if Vr{s,x,x,) G G''^-^^i[0,T] x R^) and ^,^,£^,^^ are also 
continuous, then 



BV 

Piit) - -^{t,X*{t),X*,{t))q{t), Vt G [s,T],a.s., 



hit) 



d^Vr 

8x^ 



BVr 



dVr 



{t,X*{t),X*{t))a*{t) + -^(t,X*{t),Xl{t))-l-{t)) 
a.e.t £ [s,T],a.s., 



lit), 



P2{t) = -^{t,X*{t),Xl{t))q{t), yt e [s,T],a.s., 

OXi 



(3.2) 



k2{t) = 



B^Vr 



BVr 



QJ* 



BxBx,^^'^*^^^'^'^^^^"*^^^ ~^ 8^^^^'^*^^'^'^'^^^^~t^^^ 



q{t), 



q{t) = exp 



'Bf* 1 



a.e.t e [s,T],a.s., 

2 



ar 

Bz 



(r) 



dr>, Vi e [s,T],a.s., 



where for notational simplicity, we have used in this section, for all t G [s,T], 

{b*{t) = h(t,X*(t),Y*(t),Z*(t),u*{t)), 
a*{t)=a{t,X*it),Y*it),Z*{t),u*{t)), 



or 

By 

or 

. Bz 



it) 
(t) 



5// 



BVr 



{t,X*{t),X*{t),~Vr{t,X*{t),Xl{t),^it,X*it),X*{t))a*it),u*{t) 



a// 



BVr 



{t,X*{t),X*{t),~Vr{t,X*{t),X*{t),^it,X*it),X*{t))a*it),u*{t) 
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For simplicity, in this section we write X = X"-"^'" and X* — X*'"^'" , etc. 

Proof. By the generahzed dynamic programming principle (J2.7I) . it is easy to 
obtain that 

Vrit,X*{t),X*{t)) = -Y*{t) 



-E" 



f{r,X*{r),X*,{r),Y*(r),Z*{r),u*{r))dr 



-0(X*(r),X*(T)) 



, te[s, T] 



Define a square-integrable J'^^'-martingale (recall that s G [0,r) is fixed) 



m{t) 



f{r,X*ir),X*ir),Y*{r),Z*ir),u*ir))dr 
^{X*{T),X*{T))T:], te[s,T]. 



Thus, by the martingale representation theorem (see, for example, Yong and Zhou 
[53]), there exists a unique M{-) € L'^{[s,T];R) such that 

m{t)^m{s)+ I M{r)dW{r)^V{s,x,xi)+ f M{r)dW{r), te[s,T]. 
So 

Vit,X*it),Xtit))^ - [ f{r,X*{r),X*{r),Y*{r),Z*{r),u*{r))dr 

M{r)dW{r) +Vr{T,X*{T),Xl{T)). 

On the other hand, applying the Ito's formula (Lemma 2.3) to Vr{t, X*{t), X^{t)), 
we obtain 

dVrit,X*{t),X*{t)) 

r)V BV 

-^{t,x*{t),xut)) + b*{t)^{t,x*{t),x*{t)) 

+ la*\t)^it,X*it),XUt)) 

-XS \A* 



dVr 



+ {X*it) - XXlit) - e-^'x;{t))-^it,X*{t),X*it)) >dt 



dxi 



8V 

a*{t)^{t,X*{t),X:{t))dW{t). 



Comparing the above two equations, we conclude that 

( riV r)V 

-^{t,X*{t),Xl{t)) + b*{t)^{t,X*{t),Xl{t)) 

+ \a*\t)^{t,X-*{t),Xl{t)) 

BV 

+ {X*{t) - XX* (t) - e-^'x;{t))^^{t,x*{t),x*{t)) 

^f{t,X*{t),X*{t),Y*{t),Z*{t),u*{t)), yte[s,T],a.^ 

dV 

a*{t)—^{t,X*{t),Xl{t)) = M{t), a.e. t e [s,T],a.s. 



(3.3) 
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However, by the uniqueness of solution to BSDDE (|1.5p . we have 



Y*{t)^-Vr{t,X*{t),Xl{t)), 

dV 

Z*{t) = —^{t,X*{t),Xl{t))a*{t), a.e. t G [s,T],a.s. 



Since Vr{s, x,xi) e C^'^'^ ([0, T] x R^), it satisfies the generahzed HJB equation (|2.5 
which imphcs p.ip . Also, by (I2.5p . we have 



0= - 



dVr 



-^{t,X*{t),X*{t)) + G{t,X*{t),X*{t),X;{t),u*{t),-Vr{t,X*{t),X*,{t)), 



f(t,x*w,xr(t)),-— 



^'^''(i,x*(t),xr(t)),-|^(i,x*(t),xr(t)) 



> - 



dVr 

~dt 



{t,x,Xl{t)) + G[t,x,Xl{t),X;{t),u,~Vr{t,x,Xl{t)), 

dV d'^V dV \ 

^(i,x,X*(i)),-^(i,x,X*(t)),-g(t,x,xr(i))), Vxe 



Consequently, if Vr G C^^^'^([0,r] x R^) and §7^, £.i^ are continuous, then 



dtdx ' dxdxi 



d 



dVr 

dx \ dt 
dVr 
dx 

= 0, Vi e [s, T 



— j -^(i,a:,X*(t)) + G(i,a:,X*(i),X*(t),u,-K(t,a;,X*(i)) 



^(i,X,Xi*(i)),-^(i,.T,Xr(t)),-^(i,X,X*(f)) 



dxi 



=x-(t) 



This is equivalent to (recall the definition of G (12.6 



= ^^it,x*it),x*it)) - ^(t,X*it),X*,it))b*it) 



dtdx 



dx"^ 



dV dh* 1 d^V 

— {t,X*{t\Xl(t)) — {t) - -a*^(t)-^{t,X*{t),Xt{t)) 

da* d'^V dV 

-Q^it)^it^X*(t),Xl(t))a*(t) - ^^it,X*(t\Xlit)) 



{X*{t) - AX*(i) - e-^^X;{t)) 



d^Vr 

dxdxi 



{t.x-{t),x;{t)) 



l^("-fwf<'-'^-<"-'^.-"» 



df* 



d^Vr 



dx^ 



{t,X*{t),Xl{t))a*{t) 



^(i,x*w,xr(t))^(t) 



Vie [s,r]. 
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On the other hand, applying the Ito's formula to ^- {t, X* {t) , X^ (t)) , we get 

BV 

d^{t,X*{t),Xl{t)) 

= \^^{t,x*{t),xi{t)) + ^(t,x*it),xiitw{t) 

1 R^V 

+ ^cr*\t)-^{t,X*{t),Xl{t)) 



{X*{t) - \Xl{t) - e-^^X*{t)) 



d'^Vr 

dxdxi 



it,x*it),xut))Ut 



92 V 



2 {t,X*{t),Xl{t))a*it)dW{t) 



dx 

BV Bh* Brr* B'^V 

-^{t,X*{t),Xl{t)) — [t) - —{t)^{t,X*{t),Xl{t))<y*{t) 

BV Bf* Bf* BV 



df* 



B^V BV Btj* 

-^{t,X*{t),X*{t))<j*{t) + -^it,X*{t),X*{t)) — {t) 



dt 



B^Vr 

8x^ 



{t,X*{t),X*(t))a*(t)dW{t). 



Applying again the Ito's formula to ^^{t, X* (t) , X^ {t))q{t) , noting that 



^{T,X*{T),X*{T)) = -^{X*{T),X*iT)), 



we have 



BV 

d^^{t,x*{t),x:{tMt) 

Bh* BV 

^it)^it^x*{t),xutMt) 






B'^V Bf* BV 

-^{t,X*{t),X*{t))a*{t) + -I-it)^{t,X*{t),X*{t)) 



lit) 



- g(t,X*(i),X*(t))g(i) + ^(i)g(i)|dt 

B^V Bf* BV 

^{t,X*{t),X*{t))a*{t) + ^{t)^{t,X*{t),X*{t)) 



q{t)dW{t). 



Hence, by the uniqueness of the solution to the pi{t) part of adjoint equation p.l2p . 
we have 



BV 

Piit) = ^it,X*it),X*,it))q{t), Vi e [s,T],a.s. 
Bx 



ki{t) = 



B^Vr 

Bx^ 



BVr 

Bx 
a.e.t G [s,r], a.s. 



BV Bf* \ 

it,X*{t),X*{t))a*{t) + -^it,X*{t),X*{t))-l-{t)) 



lit), 



(3.4) 
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Similarly, by (|2.5p we have 



0= - 



> - 



dV 

-^{t,X*{t),X*{t)) + G{t,X*{t),X*{t),X;{t),u*{t),-Vr{t,X*{t),X*{t)), 



dx 



dVr 



~dt 



(t,x*it),xiit)),-^{t,x*{t),xut)),-^^{t,x*{t),xi{t))) 

{t,X*(t),xi) + G{t,X*{t),xi,X;{t),u,^Vr{t,X*{t),xi), 

dV d'^V dV \ 

-^{t,X*{t),xi),~^{t,X*{t),x,),-^^{t,X*{t),x,)), Vxi e] 



Consequently, if K G C^'^''^{[0,T] x R^) and ^^, g^g^ are continuous, then 



g^^ , g^'-it,X*it),xi) + G{t,X*{t),xuX;{t),u,-Vr{t,X*{t),xi), 
-^{t,X*{t),x,),-^{t,X*{t),x,),-^^{t,X*{t),xi)) 



xi=X'{t) 



= 0, Vie [s,T]. 



This is equivalent to (recall the definition of G (|2.6 



^'^'- it,x*{t),x;it)) - p^{t,X*{t),Xt{t))b*{t) 



dtdxi 
dV 



dxdxi 



Bh* 1 B^V 

8tj* 8'^V r)V 

-g^^{t)^{t,X*{t),X:{t))a*{t) + X^^{t,X*{t),X:{t)) 

- {X*{t) - \X*{t) - e-^'x;{t))^{t,X*{t),X*{t)) 
8f* Bf* BV 



f<« 



BxBxi 



{t,X*{t),X*{t))a*{t) 



BV Bfj* 

+ ^it,x*{t),xm)^it) 

ox ox I 



, yte[s,T]. 
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On the other hand, applying the Ito's formula to ^^{t,X*{t),X^{t)), we get 
dg(t,X*(i),X*(t)) 

+ {X*{t) - XX*,{t) - e-^'x;{t))^{t,X*{t),X*it))\dt 



dx\ 



dxdxi 



{t,X*{t),Xl{t))a*{t)dW{t) 



8V 8h* 8a* 8'^V 

8V 8f* 8f* 8V 



df* 

d^Vr 



8^V 8V 8a* 

^it,xmixmy%t) + ^it,x*it),xm)^w 



{t,X*{t),Xl{t))a*{t)dW{t) 
av,. 

dVr 



dx8xi 
Applying again the Ito's formula to ^{t, X* (t), Xl{t))q{t), noting that 



g^^{T,X*iT),X*,{T)) = -^^iX*iT),X*,{T)), 



dt 



we have 



8V 

di^^{t,X*{t),X*it))q{t) 

lf^itf-^it,x*it),x*AtMt) 



da 
8xi 



* r/^2 



(t) 



8^Vr 



df* . . dVr 



^^^■{t,X*{t),Xl{t))a*{t) + -^(t)^{t,x*{t),Xl{t)) 



lit) 



^^{t,X*{t),Xl{t))a*{t) + ^it)^it,X*it),X;{t)) 
8xoxi oz ox I 



q{t)dW{t). 



Hence, by the uniqueness of the solution to the P2{t) part of adjoint equation (|2.12p . 
we have 



P2{t) = ^^{t,X*{t),Xl{t))q{t), Vt G [s,T],a.s. 



k2{t) = 



d^Vr 



dVr 



QJ. 



{t,X*{t),Xl{t))a*{t) + -^{t,X*{t),Xl{t))-^{t) 
8x8xi oxi 8z 

a.e.t e [s, T], a.s. 



q{t), 



(3.5) 
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And finally 



g(t)=cxp<j / —{r)-- 



oz 



2 



dr\, yte[s,T],a.s., (3.6) 



can be easily obtained by solving the forward equation of q{t) directly. The proof is 
complete. D 

4. Application to Consumption and Portfolio Optimization with Re- 
cursive Utility and Bounded Memory. In this section, we discuss a consumption 
and portfolio optimization problem with recursive utility and bounded memory in the 
financial market. The financial framework in this problem is initiated introduced by 
Chang et al. [4], with classical cost functional. In this paper, we generalize their model 
to the case with recursive utility. The optimal portfolio and consumption strategies 
are obtained by both dynamic programming and maximum principle approaches, in 
the meanwhile the relations we obtained in Theorem 3.1 are illustrated. 

Let us first describe the environment of the financial market. Consider an investor 
who can invest his money into a risky asset and a riskless asset. The risky asset can 
be a stock, a mutual fund, etc. The riskless asset earns a fixed interest rate r > 0. 
We can treat the money invested on the riskless asset as money deposited into a bank 
account. We assume that the investor can consume his/her wealth. 

Let U{t) be the amount invested on the risky asset and V{t) is the amount invested 
on the riskless asset. The total wealth is given by X{t) = U{t) + V{t). We consider 
the situation in which the performance of the risky asset has some memory (delay). 
Because many investors will look at an asset's past performance before they invest 
their money on the asset, the increasing investment performance of their wealth in 
the past tends to drive the investors to invest more on the risky asset, hence it can 
push the price of the risky asset even higher. On the other hand, if the price has been 
decreasing a lot, investors tend to sell the asset and invest on other assets, which will 
drive the price to go down further. To describe this phenomenon, we assume that 
the performance of the risky asset depends on the following delay variables Xi (t) and 
X2{t): 

Xi{t)^ e^^X{t + T)dT, X2(t)^X{t-6), te[s,T], (4.1) 

for any initial time s e [0,T). Here A is a constant and (5 > is the delay parameter. 
The parameter S gives us the duration of the past that the investor usually cares 
about. 

Let {W{t),t > 0} be a one-dimensional standard Brownian motion defined on a 
complete probability space (J7, J^, P). We assume that the filtration J^ ~ a{W{T); < 
T < t} is augmented by all the P-null sets in J^. 

We assume that U{t) and V{t) follow the stochastic differential equations: 

dU{t)^ [tioU{t)+niXi{t)+H2X2{t)]dt + aU{t)dW{t), (4.2) 



dV{t)=[rV{t)-C{t)]dt, (4.3) 

where fJ^o, fJ-i, fJ-2 and a are positive constants, and C(t) is the consumption rate. 
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Add them together, and use the fact that X{t) — U(t) + V{t), then we get the 
equation for the wealth X{t): 

' dX{t) = [iioU{t) + HiXiit) + M2^2(i) + rV{t) - C{t)] dt 
< +aUit)dW{t), te [0,T], (4.4) 

^ x{t) = ip{t), tGM,o], 

where continuous function ip : [—6, 0] — > R is the initial condition for information 
about X{t) fori 6 [-6,0]. 

Further, instead of U{t) and C(t), we use c{t) = C{t)/X{t) and u{t) = U{t)/X(t) 
as our consumption and portfolio control, respectively (note that X{t) > 0, a.s. is 
proved in Lemma 2.2 of Chang et al. [1]). It is easy to see that V{t) ~ X{t) — U{t) = 
X{t){l — u{t)). Now we can rewrite the equation for X{t) as 

dX{t) = [{{no - r)u{t) - c{t) + r)X{t) + ^llXl{t) + A*2^2(t)] dt 

+ au{t)X{t)dW{t), t<^[s,T], (4.5) 

X{t) = Lp{t) >0, t e [-5,0]. 

Now we define the admissible control space 11 for the control variables u{t) and 
c{t): 

Definition 4.1. (Admissible Control Space) A control strategy {u{t), c{t)) is said 
to he in the admissible control space 11 if it satisfies the following conditions: 

{i) {u{t),c{t)) is J- 1- adapted processes; 

{a) c(t) > 0,Vt e [0,T]; 

{Hi) For any t G [0,r], we have 

\u{t)X{t)\ < Ai\X{t) + fi2Xi{t)\, \c{t)X{t)\ < A2\X{t) + /i2Xi(i)|, 

where Ai, A2 > are positive constants. 

The investor wants to minimize the following recursive utility 

Jr{u{-),c{-)):=-Y{t)l^^, (4.6) 

over the admissible control space 11, where 



(4.7) 



-dY{t) = [~ /3Y{t) + -{c{t)X{t)y]dt - Z{t)dW{t), te [0,T], 

Y{T) ^ -{X{T) + 9Xi{T)y . 

7 

and /3 > 0, 7 £ (—00, 1), 7 7^ 0, 6* £ M are constants. 

Remark. The recursive utility functional defined in (14. 7p with generator 

f{t,x,xi,y,z,u,c) = -I3y + -{cx^ , (4.8) 

7 

stands for some standard additive utility of recursive type. It can be easily checked 
that / defined above is concave with respect to (c, y) and increasing with respect to 
c, which are classical properties that utility functions must satisfy. Recursive utility 
like (|4.8p is meaningful and nontrivial generalization of the classical additive utility 
and has many applications in mathematical economics and mathematical finance. 
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For more detail about recursive utilities, see DufSe and Epstein [7|, Section 1.4 of El 
Karoui et al. [S] or Schroder and Skiadas [25] , 

Since we are going to involve the dynamic programming principle, we will adopt 
the formulation as in Section 2. For given (s, ip) e [0, T) x C([— (5, 0]; R) and admissible 
control (li(-), c(-)), the wealth equation is 

' dX^'^-^^'^'it) = { [(mo - r)u{t) - c{t) + r\X''^-^^'^'{t) + ^i^Xl^'^-'^^^t) 

^ x^'^-^-'^{t)^v{t)>Q, ieM,o]. 

and the recursive utility functional is defined as 

J.(s,(^;u(-),c(.)) = -r^^^^"'^(i)|,^,, 
where 

_^ys,v;u,cf^^^ = I - /3r"''^'"''=(t) + -{c{t)X'^'^''''''{t)y\dt 



(4.10) 



(4.11) 



ys,V';u,c^y^ ^ i(X^^'^'"^ = (r) +6IX"''^'"''=(T))''. 



This problem can be reformulated as follows. The state process (X''''^'"''^(-), y*''^;"''=(.)j 
^s,ip;«,cj-.-j-j q£ q^j. sys^g]-Q js described by the following coupled FBSDDE 

dX'-^-'^--{t) = { [(mo - r)u{t) - c{t) + r] X^^'^^«'=(t) + MiXr'^^"'^(i) 
+ M2^2''''"''W}^< + ffM(t)^''^'"''(t)rfW^(i), 

_^ys,v;«,c(^) = I - /3y''''^'"''=(i) + -{c{t)X'^'P'''^''[t)y\dt 

-Z'-'^ •''■'' {t)dW{t), te [s,T], 
X''-'P'''-'''{t) ^ ip{t) >0, te [s~5,s], 



(4.12) 



1 



=(r) = -(x^^'^'"^^(T) + 6iXi"''^'"''=(r))'^, 



and the cost functional is given of the form 



Jris,ip;u{-),c{-)) = ~Y^-'^-'^-%s) 



Tos,(/:;u,c 



PY'^,V■,u,c^^^ ^ l(c(i)X''^'^'^'"(t))^ 

7 



+ -(X''''^'"^^(T) + 6lXf''^^"''=(r))' 
7 



dt 



(4.13) 



The consumption and portfolio optimization problem is to find an admissible 
{u* {■) , c* {■)) such that 

Vris,ip) = Jr{s,ip;u*{-),c*{-))= cssiuf Jris,ip;u{-),c{-)), (4.14) 

(«(•). c(-))en 



for all (s,(p) G [0,T) x C([-(5,0]; 
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We can see that we are now in the finite dimensional space such that the value 
function Vr only depends on {s,x,Xi), i.e., 



where Vr : [0,r] 



Vr{s,(p) = Vr{s, X,Xl,X2) = Vr{s,X,Xl), 

|2 -^ M, and 

x = x{ip) =(p{0), 



(4.15) 



y = y{f) = / e '^ip{T)dT. 

Js 

And all the assumptions in Section 2 are satisfied. Then we can use both dynamic pro- 
gramming (Theorem 2.4) and maximum principle (Theorems 2.5 and 2.6) approaches 
to solve this problem. 

4.1. Dynamic Programming Approach. In this subsection, we solve the 
above consumption and portfolio optimization problem, applying Bellman's dynamic 
programming approach. The main mathematical tool is Theorems 2.4, that is, the 
generalized HJB equation (12. 5|) . 

In this case, the generalized HJB equation (|2.5p that the value function of the 
form V{s,x,xi) should satisfy, reduces to 

dVr, , fl 2 2 29^Vr, , , , ^ ^K , , 

-—-{s,x,xi) = ™ax<^ -a u X -j—(s,x,xi) + [fio ~ r)ux——{s,x,xi) 

/ ^^W ^ ^ 1 7 7 

-|-max< —ex— — [s,x,xij-\ — c'x' 

c>o I ax 7 

,9Vr, 

+ [rx + ^IiXi + ^i2X2)^7—[S,X,Xi) 

ox 



dVr 



+ {x - Xxi - e X2)^ — {s,x,xi) - l3Vr{s,x,xi), Va;2 e M, 



dxi 



(4.16) 



Vr(T,X,X2) — (x + 9xi) , 

7 
By (j4.16p . the candidate for the optimal consumption and portfolio strategies is 

u*{s) = u*{s,x,xi) — — - 



(/^o -'')^('S>2;,a;i) 



c*{s) = C*{s,X,Xi) = 

Plug these into (|4.16p . we have 

It/ 

f3Vr{s,X,Xi) - — ^(s,X,Xi) 



(T^X-Q^{s,X,Xi) 



1 i^,„-rr[^is,x,x,f ,(l_A\m, . 

2 a^^-^{s,x,x,) +W ^J[fe^^'"'"^^. 



dVr 



+ {rx + ^ilXl + fl2X2)-^is,X,Xi) 

dV 

+ (x — Xxi ~ e~''^^X2)Tr-^{s,x,xi), Vx2 G K, 
ox I 
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It can be rewritten as 

dVr 



OS 

2 cr^7J^(s,a;,a;i) 



- 1 



7 



\dVr 

dx 



{s,x,xi) 



+ (rx + pi\x\)---\s,x,xx) + {x - \xi)- — (s,x,xi), 



+ 



^2-5— (s,x, Xi) -e (s,x,xi) 



dx 



dxi 



X2, Vx2 e M, 



(4.17) 



Motivated by the terminal condition in (|4.16|) . we try a value of the form 



Vr{s,x,xi) — Q{s)(x + 6xi) , 



(4.18) 



where Q{-) is some differentiable deterministic function with Q{T) = 1. We then have 

-—^{s,x,xi)^ Q'{s)(x + exiy, 

as 7 



dVr 
dx 

dx"^ 

dxi 



7-1 



{s,x,xi) = -Q{s){x + 9xi) , 
{s,x,xi)= -{j -l)Q{s){x + Oxi) 



7-2 



{s,x,xi)= -9Q{s)[x + 9xi) 



7-1 



Put them into (|4.17p . we obtain that 
1 



7 



/30(s) - g'(s) 



(x + Oxi) 



+ {(r + 0)2: + (^1 - X9)xi + {n2 - e-^^e)x2^Q{s){x + 6x1^, Vxa 



Assume that 



^26^^ M2 > 0, 



(4.19) 
(4.20) 



holds, we can rewrite (|4.19p as 



1 



PQ{s)~Q'{s) {x + fi2e^^xi) 



„A(5„ \7 



(4.21) 



\^(^^„..^KA'' (4-22) 



+ (r + /i2e^^)Q(s)(:r + ^26^^2:1)^. 
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Canceling the term (x + fi2e^^xi^ on both sides, we can get 

1 



1 

7 

That is 



(3Qis) ~ Q'is) 



^^Q(=s) (± - 1 j [Qis)] ^ + ir + ^.e^^)Q(.). 



where 



r Q'{s) = (7 - 1) [Q(s)] ''-' + AQ{s), s G [0, T], 

1 0(r) = 1, 



^-^+i^-*+'-^')>°- 



(4.23) 



(4.24) 



(4.25) 



By some elementary technique of solving ODEs, we can obtain the following explicit 
solution 



Q{s) = 



1 — 7\ A(T-3) 1—7 



1-7 



6[o,r]. 



(4.26) 



We have proved that 

Theorem 4.2. The function Vr{s,x,xi) given by (RTT^ is a classical solution to 
the generalized HJB equation OTT^, and it is equal to the value function defined by 
[jA5d. In addition, the optimal portfolio and consumption strategies are given by 

u (s) = u (s, X (s), Xi (s)) = (1- )o-^X*(s) ^' * f ' ^' *- '^ 



c*(5)^c*(.,x*(s),xr(5)) = ^^itoj^|^^Ii^Q(.)^, .e[o,r], (4.28) 



where (X*(-), X*(-)) is the solution to SDDE C7^ with respect to (u*(-), c*(-)), and 
Q(-) satisfies the ODE OT^l), which admits the explicit solution \^.26^ . 

4.2. Maximum Principle Approach. In this subsection, under the assump- 
tions (|i?^ and (|MI1), we prove that (|i??fl) and (|i?^ are optimal portfolio and 
consumption strategies, respectively, applying Pontryagin's maximum principle ap- 
proach. The main mathematical tool is Theorems 2.5, 2.6, that is, the sufficient and 
necessary maximum principles. 

By (|2.1ip . the Hamiltonian function takes the form 

H{t,x,xi,X2,y,z,u,pi,p2,q,ki) =pi|[(^o ~ r)u - c + r]x + Hixi + (12X2} 



(4.29) 
For candidate optimal control strategies {u* {■) , c* {■)) and the corresponding state 



+ P2{x — Xxi — e X2) + kiaux ~ q[~ /3y -{ c'^x"'] 
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{X*{-),Y*{-),Z*{-)), the adjoint equation (f^l^ reduces to 



-dpi (t) = } [i^io - r)u* (t) - c* {t) + r]pi (t) + p2{t) + au* {t)h (t) 
~ q{t)c*{t)^X*{t)''-Hdt ~ ki{t)dW{t), 

-dp2{t) = (miPI W - \p2{t))dt - k2{t)dW{t), 

-dp3{t) = {ii2Pi{t) - e-^'p2{t))dt, (4-30) 

q{t) = -I3q{t)dt, tG[0,T], 

Pi(r) - -{x*{T) + exi{T)y-\{T), 
P2{T) = -e{x*{T) + exi{T)y'\{T), 

{ P3{T)^0, g(0) = l. 

The maximum condition ()2.16p in Theorems 2.5 and 2.6 teU us that, we can find 
u*{-) and c*(-) by maximizing 

u -^ H{t,x,Xi,X2,y,z,u,c,pi,p2,q,ki) 
and 

c -)■ H{t, X, Xi,X2, y, z, u, c,pi,p2, q, ki) 
over ah u and c > 0, respectively. Then we have 
dH 



g^-{t,x,Xi,X2,y,z,u,c,pi,p2,q,ki) 



u—u" 



and 



^*(t) (4.31) 

= (mo - r)pi{t)X*{t) + aki{t)X*{t) =. 0, 



— (i, X, xi ,X2,y, z, u, c,pi,p2,q, fci) 



C—C* 



(*) (4.32) 

= -Piit)x*it)~qit)c*ity-'x*ity^o. 

By (j4.32p . we directly achieve that 

Motivated by the terminal condition for pi (i) in (|4.30p , we try to find pi (i) of the 
form 

Pi{t) - -Q{t){X*it) + 9X*{t)y-\it), (4.34) 

where Q{-) is a deterministic differentiable function with Q{T) = 1. 
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Applying the Ito's formula to (I4.34[) . we can get 

dpiit) = -Q'{t)q{t){x*{t)+ex*{t)y'^dt 

- Q{t)q{t)h - l){X*{t) + 9X*{t)y'\dX*{t) + 9dX*{t)) 

- lQ{t)q{t)h i)(7 - 2){x*{t) + 9Y*it)y-'{dx*{t) + edxi{t)f 

+ pQ{t)q{t){X*{t) + eY*{t)y'^dt 

-{x*{t) + exi{t)y-'Q'{t)q{t) 
-{x*(t) + exi(t)y-\^-i)Q{t)q{t) 

X ((mo - r)u*{t) - c*{t) + r)X*{t) + ^ilXl{t) + ii2X;{t) 

+ e[x*{t) - \xi{t) - e-^^x^{t)\ 

- \{X*{t) + eXl{t)y-\^ 1)(7 - 2)a'Q[t)q{t)u*\t)X*\t) 

+ {x*{t) + exi{t)y'^pQ{t)q{t)dt\dt 

- {X*{t) + eXl{t)y'^{-i - l)aQ{t)q{t)u*{t)X*{t)dW{t). 

Comparing the coefficient of dW{t) in ()4.35p with that in (|4.30p . we obtain 

ki{t)^-{X*{t)+6Xl{t)y'^{-1~l)aQ{t)q{t)u*{t)X*{t), te [0,r]. (4.36) 

Next, by the adjoint equation (|4.30p . the condition (|2.15p that P3{t) — 0,Vi € 
[0,r] can be formulated as follows: 

^i2Plit)~e-^^P2it)=o, vte [o,r]. 



(4.35) 



That is 

P2{t) = /i2p/V(0 = -^i2e^^Qit){X*{t)+0X*it)y'\it). 
By the terminal conditions in (|4.30p . we must have 

e = ^i2e^^. 

That is 

pi(t) = -Qit){X*{t) + ti2e^'Xl{t)y-\{t), 
and 

P2{t) ^ fi2e^'pi{t) ^ ~fi2e^'Q{t){X*{t) + fi2e^'X*{t)y'\ 
Consequently, we obtain 

fci(i) = -{X*it) + /i2e^^X*(t))^"'(7 - l)aQit)qit)u*it)X*it), 
and 

k2it) = -fi2e^^ki{t) 
- -fi2e^'{X*{t)+^i2e^'X*{t)y-^{j-l)<jQ{t)q{t)u*{t)X*{t). 



(4.37) 
(4.38) 
(4.39) 
(4.40) 
(4.41) 
(4.42) 

(4.43) 
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Putting (lOOl and (P^i^ into (|i3T|) and (|I35)) . we can get 



c*(t) = ''*^^^+//(^''''^^^^^ [QW]^,te[0,T]. (4.45) 

Next, comparing the coefficient of dt in (|4.35p with that in (|4.30p . if the fohowing 
assumption 

/ii - \fi2e^^ = {r + H2e^^)fi2e^^ (4.46) 

holds, we obtain that 

^'(*)= [/5+^^^(^^-7(^ + M2e^')]g(t) + (7-l)[Q(i)]^,ie[0,T], 

QiT) = 1, 

(4.47) 
which is exactly the same ODE (|4.24p in Theorem 4.3, and the explicit solution to it 
is (14261) . 

We have proved that 

Theorem 4.3. Suppose that 114. 3 9\ ) and !Ji4-4(^ hold. Then the optimal portfolio 
and consumption strategies are given by 

.•W . ,r«„Y.(0,AT,)) ^ "'°~^',',!'^)t:;,"!(Q ■"'"" . ' e |0,T,. (4.48) 

e-(t) 5 c-(t.x-(t).x;(ty, = -^''W ^/.',^"^'''" [0(0] ^. felo.T], (4,49) 



where {X*{-),X^{-)) is the solution to SDDE C7^ wjf/i respect to {u* {■) , c* {■)) , and 
Q{-) satisfies ODE |^./^7p , which admits the explicit solution \4.2(j^ . 

4.3. Relationship and Some Discussions. In addition, the relations in The- 
orem 3.1 can be easily verified. In fact, relationship (j3.ll) is obvious from (|4.16p . And 
the relations given in ([3^ can be easily obtained from (|4J8l) . (|440| . (|OT|) . (|442|) . 
(14:431) and q[t) = e'^^^t £ [0,T]. 

From the condition (|4.46p . we can get 

^ll = \^i2e^^ + {r + ^i2e^^)^i2e^^ = M2e^''(A + r + ^26^'^). (4.50) 

So is is easy to see that /ii = if and only if /i2 = 0, provided that fJ-2 ^ and 
lim /ii = cx). In other words, the price dynamics of X{t) must depend on both 

Xi(t) and X2{t) at the same time, in order to obtain an explicit solution of V, u* and 
c* in a finite dimensional space. Otherwise, when ^2=0 (then /ii =0), that is, the 
dynamic equation of X(t) does not depend on Xi(t) and X2{t) explicitly, our model 
reduces to the consumption and portfolio optimization model with recursive utility 
but without delay. 
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5. Conclusion. Pontryagin's maxinium principle and Bellman's dynamic pro- 
gramming, the most important two different approaches to study deterministic and 
stochastic optimal control problems which are systematically discussed in Yong and 
Zhou [33], are also two main tools to deal with stochastic control problems with time 
delay. Both of these two approaches have been developed separately and indepen- 
dently. In this paper, we have initiated to discuss the relationship between maximum 
principle and dynamic programming for one kind of stochastic recursive optimal con- 
trol problems with time delay, when only the pointwise and distributed time delays 
in the state variable is considered. One advantage for this kind time delay is that the 
corresponding generalized HJB equation is finite dimensional, under some suitable 
conditions on the coefficients. Sufficient and necessary maximum principles are also 
proved. Under the assumption that the value function is enough smooth, we obtained 
its relations with the adjoint processes and the generalized Hamiltonian function. A 
consumption and portfolio optimization problem with recursive utility and bounded 
memory in the financial market, was discussed to show the applications of our re- 
sult. Explicit solutions for optimal portfolio and consumption strategies in a finite 
dimensional space derived by the two different approaches, coincide. 

While our work makes a substantial initial step in the studying of the relationship 
between Pontryagin's maximum principle and Bellman's dynamic programming ap- 
proaches to the stochastic recursive optimal control problems with time delay, more 
remains to be done. Potential extensions of the present work include the consideration 
of the problem of stochastic delayed system under model uncertainty in Pamen [24j . 
This will be considered in our future research. 



REFERENCES 

[I] N. Agram, S. Haadem, B. 0ksendal and F. Proske, A maximum principle for infinite horizon 

delay equations, Preprint in Pure Mathematics, University of Oslo, Oslo, Norway, June 2012. 
[2] N. Agram and B. Oksendal, Infinite horizon optimal control of forward-backward stochastic 

differential equations with delay, arXiv.org/abs/1302.1723vl [math.OC], February 2013. 
[3] M. H. Chang, T. Pang and M. Pemy, Optimal control of stochastic functional differential 

equations with a bounded memory, Stochastics: An International Journal of Probability and 

Stochastic Processes, 80(2008), pp. 69-96. 
[4] M. H. Chang, T. Pang and Y. P Yang, A stochastic portfolio optimization model with bounded 

memory. Mathematics in Operations Research, 36(2011), pp. 604-619. 
[5] L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay 

and application, Automatica, 46(2010), pp. 1074-1080. 
[6] L. Chen and Z. Wu, Dynamic programming principle for stochastic recursive optimal control 

problem with delayed systems, ESAIM: Control, Optimisation and Calculus of Variations, 

18(2012), pp. 1005-1026. 
[7] D. DuFFiE AND L. G. Epstein, Stochastic differential utility, Economctrica, 60(1992), pp. 353- 

394. 
[8] N. El Kargui, S. G. Peng and M. C. Quenez, Backward stochastic differential equations in 

finance. Mathematical Finance, 7(1997), pp. 1—71. 
[9] N. El Karoui, S. G. Peng, and M. C. Quenez, A dynamic maximum principle for the optimiza- 
tion of recursive utilities under constraints. The Annals of Applied Probability, 11(2001), 

pp. 664-693. 
[10] I. Elsanosi and B. Larssen, Optimal consumption under partial observations for a stochas- 
tic system with delay. Preprint in Pure Mathematics, University of Oslo, Oslo, Norway, 

December 2001. 

[II] I. Elsanosi, B. Oksendal and A. Sulem, Some solvable stochastic control problems with delay. 

Stochastics & Stochastics Reports, 71(2000), pp. 69-89. 
[12] S. Federico, a stochastic control problem with delay arising in a pension fund model. Finance 

& Stochastics, 15(2011), pp. 421-459. 
[13] M. Fuhrman, F. Masiero and G. Tessitore, Stochastic equations with delay: optimal control 



34 STOCHASTIC RECURSIVE DELAYED OPTIMAL CONTROL 

via BSDEs and regular solutions of Hamilton- Jacobi-Belhnan equations. SIAM Journal on 
Control and Optimization, 48(2010), pp. 4624-4651. 

[14] J. H. Huang, X. Ll and J. T. Shi, Forward- Backward linear quadratic stochastic optimal 
control problem with delay, Systems & Control Letters, 61(2012), pp. 623-630. 

[15] J. H. Huang and J. T. Shi, Maximum principle for optimal control of fully coupled forward- 
backward stochastic differential delayed equations, ESAIM: Control, Optimisation and Cal- 
culus of Variations, 18(2012), pp. 1073-1096. 

[16] V. B. KOLMANOVSKII AND T. L. Maizenberg, Optimal control of stochastic systems with af- 
tereffect. Stochastic Systems, Translated from Avtomatika i Telemekhanika, 1(1973), pp. 
47-61. 

[17] V. B. KOLMANOVSKII AND L. E. Shaikhet, Control of Systems with Aftereffect, Translation of 
Mathematical Monographs, 157, American Mathematical Society, 1996. 

[18] B. Larssen, Dynamic programming in stochastic control of systems with delay, Stochastics & 
Stochastics Reports, 74(2002), pp. 651-673. 

[19] B. Larssen and N. H. Risebro, When are HJB-equations in stochastic control of delay systems 
finite dimensional?. Stochastic Analysis and Applications, 21(2003), pp. 643-671. 

[20] S. E. A. Mohammed, Stochastic Functional Differential Equations, Pitman, 1984. 

[21] S. E. A. Mohammed, Stochastic differential equations with memory: theory, examples and ap- 
plications, Stochastic Analysis and Related Topics VI. The Geido Workshop, 1996, Progress 
in Probability, Birkhauser, 1998. 

[22] B. 0KSENDAL AND A. SuLEM, A maximum principle for optimal control of stochastic systems 
with delay, with applications to finance, J. M. Menaldi, E. Rofman, A. Sulem, eds.. Op- 
timal Control and Partial Differential Equations-Innovations and Applications, lOS Press, 
Amsterdam, 2000. 

[23] B. 0KSENDAL, A. Sulem and T. S. Zhang, Optimal control of stochastic delay equations and 
time-advanced backward stochastic differential equations. Advances in Applied Probability, 
43(2011), pp. 572-596. 

[24] O. M. Pamen, Optimal control for stochastic delay system under model uncertainty: a stochastic 
differential game approach. Preprint in Pure Mathematics, University of Oslo, Oslo, Norway, 
September 2011. 

[25] S. G. Peng and Z. Yang, Anticipated backward stochastic differential equations. The Annals 
of Probability, 37(2009), pp. 877-902. 

[26] M. Schroder and C. Skiadas, Optimal consumption and portfolio selection with stochastic 
differential utility. Journal of Economic Theory, 89(1999), pp. 68-126. 

[27] J. T. Shi, Maximum principle of recursive optimal control problem for forward-backward 
stochastic delayed system with poisson jumps, Scientia Sinica Mathematica, 42(2012), pp. 
251-270. 

[28] J. T. Shi, Two differential approaches to stochastic control problems with delay and applications, 
Submitted manuscript, 2013. 

[29] J. T. Shi and Z. Wu, Maximum principle for forward-backward stochastic control systems with 
random jumps and applications to finance. Journal of Systems Science and Complexity, 
23(2010), pp. 219-231. 

[30] G. C. Wang and Z. Wu, The maximum principle for stochastic recursive optimal control 
problems under partial information, IEEE Transactions on Automatic Control, 54(2009), 
pp. 1230-1242. 

[31] Z. Wu, A general maximum principle for optimal control problems of forward-backward stochas- 
tic control systems, Automatica, 49(2013), pp. 1473—1480. 

[32] Z. Wu AND Z. Y. Yu, Dynamic programming principle for one kind of stochastic recursive 
optimal control problem and Hamilton- J acobi- Bellman equation, SIAM Journal on Control 
and Optimization, 47(2008), pp. 2616-2641. 

[33] J. M. YONG AND X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, 
Springer- Verlag, New York, 1999. 

[34] Z. Y. Yu, The stochastic maximum principle for optimal control problems of delay systems 
involving continuous and impulse controls, Automatica, 48(2012), pp. 2420-2432. 



